(19) 



3 



Europaisches Patentamt 
European Patent Office 
Office europeen des brevets 



(12) 



(id EP0 951 175 A2 

EUROPEAN PATENT APPLICATION 



(43) Date of publication: 

20.10.1999 Bulletin 1999/42 

(21) Application number: 99302861 

(22) Date of filing: 13.04.1999 



(51) lntCI.6: H04N 1/32 

BEST AVAILABLE COPY 



(84) Designated Contracting States: 


(72) Inventor: Wang, Shen-ge 


AT BE CH CY DE DK ES Fl FR GB GR IE IT LI LU 


Fairport, New York 14450 (US) 


MC NL PT SE 




Designated Extension States: 


(74) Representative: Skone James, Robert Edmund 


AL LT LV MK RO SI 


GILL JENNINGS & EVERY 




Broadgate House 


(30) Priority: 15.04.1998 US 60469 


7 Eldon Street 




London EC2M 7LH (GB) 


(71) Applicant: XEROX CORPORATION 




Rochester, New York 14644 (US) 





(54) Automatic detection and retrieval of embedded invisible digital watermarks from halftone 
images 



(57) A system for automatic detection and retrieval 
of embedded invisible digital watermarks retrieves dig- 
ital watermarks from halftone images. Specifically, by 
supplying an image to the system, through a process of 
autocorrelation and shifting, the embedded invisible wa- 
termark becomes visible. The process includes scan- 
ning or supplying an image to the system, calculating 



the global autocorrelation of the image, selecting a mov- 
ing window size, conducting a piecewise localized au- 
tocorrelation for each window-sized portion of the im- 
age, retrieving the embedded, initially invisible, water- 
marks, normalizing the resultant image for visualization 
and d is u laying the resultant image with the now visible 
retrieved embedded digital watermarks. 
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Description 

[0001] This invention is directed to a system and method for automatically detecting invisible digital watermarks 
embedded in halftone images, or other images that use microstructures to simulate continuous tone color or grayscale 
5 images. 

[0002] Methods for protecting copyrightable works have evolved from a simple designation below the work to highly 
complex methods for embedding watermarks in the work. Watermarking can take two basic forms: visible and invisible. 
Visible watermarks are the commonly-seen copyright logos or symbols that are generally affixed to the work before 
sales or distribution. Especially in the case of images., the presence of the watermark is very visible, and is generally 
w difficult to remove without damaging the image. Generally speaking, visible watermarks do not harm the image, even 
though the watermarks may detract form the overall esthetics of the image. Furthermore, the visible watermark is a 
potential target for fraud. Since a fraudulent copier is actually placed on notice regarding the presence of the watermark, 
it is possible to attempt to remove the visible watermark from the image. 

[0003] Invisible watermarks are far more creative and can encompass the standard and commonly used copyright 
is logos or symbols, as well as company logos, serial numbers, origin identification marks, and/or encrypted data. These 
invisible watermarks are embedded into the work in a way which is not generally discernible without the aid of a visu- 
alization device such as a key or computer. Theoretically, these embedded images can be retrieved from the work at 
any time in the work's history or from any other form or embodiment into which the work may have been translated. 
This allows the owner to track the work and clearly establish ownership rights when those right are in dispute. Further- 
20 more, since the embedded watermark image is essentially invisible to the unaided eye, the likelihood of tampering with 
or removal of the watermark is reduced. 

[0004] This invention provides a system and method for embedding and retrieving digital watermarks that overcomes 
the problems associated with recovering these marks from non-original images. 

[0005] This invention further provides a system and method that allows previously unretrievable embedded invisible 
25 watermarks to be recovered from works that have been converted from a digital format to a printed copy such as a 
print, or from a reproduction made, for example, on a photocopier. 

[0006] This invention also provides a system and method that uses localized autocorrelation to estimate the exact 
amount of the separation between two adjacent correlated halftone patterns that when properly combined produce a 
visible watermark. Localized autocorrelation of the two adjacent correlated halftone patterns can reduce the effect of 
30 distortion and nonuniformity to a minimum. Additionally, global scaling and/or rotation can be treated as individual local 
shifting and does not need global correction. Thus, localized autocorrelation generates a clearer result. 
[0007] This invention additionally provides a system and method that uses a two-step autocorrelation process to 
extract or retrieve embedded digital watermarks from a printed or copied image. 

[0008] Invisible watermark retrieval depends on the pixel-to-pixel comparison between a bitmap of a halftone image 
35 and the bitmap of the halftone image having a certain shift relative to itself. In some areas the bitmap and its shifted 
version are highly correlated, i.e., near identical, while in other areas they are uncorrelated or highly "conjugately" 
correlated, i.e.. one bitmap is the inverse of the other bitmap. The pixel-to-pixel comparison between the original and 
shifted bitmaps can provide a contrast between the correlated areas and other areas. Therefore, the embedded, or 
hidden, watermark becomes visible. 
40 [0009] However, retrieval of the original bitmaps from printed copies is not trivial, especially from high-resolution 
printed copies. Both printing and scanning processes introduce overlapping, distortion and nonuniformity, as well as 
noise, to the embedded image. The exact bitmap information in very dark regions of the image in the printed copy is 
difficult to recover. Even in the brighter regions of the image, where there is greater contrast, retrieving the digital 
watermark is expected to be successful only in a statistical sense. The spatial separation between the two adjacent 
45 correlated halftone patterns varies and the amount of shift is generally not an integer number of bitmap pixels on 
rescanned images. Accurately determining the spatial separation, or the location of a correlation peak, becomes the 
most critical requirement when detecting hidden watermarks. 

[0010] Autocorrelation is most easily visualized by imagining two transparencies containing identical images. The 
two images are then overlayed so they are in perfect alignment. At this point, the maximum amount. of light passes 

so through the images. Autocorrelation with (0. 0) arguments refers to this perfect alignment, where there is zero shift 
between the two images and a maximum amount light of light passes through the images. The value of the autocor- 
relation with arguments other than (0, 0) can be visualized as one image being shifted relative to the other image, 
where the amount of light passing through the images is reduced. Usually, the reduction in transmitted light falls quickly 
near the (0. 0) position and the autocorrelation becomes approximately constant when its arguments, viewed as the 

55 relative shift between the two transparencies, are large. However, if the image contains spatially periodic structures, 
such as the halftone image generated by tiling a single halftone screen over the entire image, relative peaks of auto- 
correlation occur for certain arguments. These relative peaks may be visualized as the amount of light transmitted 
through the two transparencies for a certain relative shift. The relative peak amount of transmitted light may not be as 
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great as the primary, or absolute, peak amount of transmitted light that occurs when the images are perfectly aligned. 
However, this secondary relative peak is detectable. Therefore, if watermarks are embedded in a halftone image, i.e., 
in the periodic structure of the image : autocorrelation repeats itself to an extent when the periodic occurrences of the 
watermark are themselves aligned between the two images. 
5 [0011] Therefore, by using a two-step autocorrelation process, the system and method of this invention enables 
recovery of invisible digital watermarks from printed copies. 

[001 2] These and other features and advantages of this invention are described in or are apparent from the following 
detailed description of the preferred embodiments. 

[001 3] The preferred embodiments of this invention will be described in detail, with' reference to the following figures. 
10 wherein: 



Figure 1 is a halftone image containing an invisible or embedded digital watermark: 

Figure 2 shows the embedded watermark retrieved by using the method according to this invention: 

Figure 3 is a functional block diagram of a watermark detection device according to this invention: 

is Figure 4 is a functional block diagram showing the autocorrelator of Fig. 3 in greater detail: 

Figure 5 is a functional block diagram outlining in greater detail the global autocorrelation determiner of Fig. 4: 
Figure 6 is a functional block diagram outlining in greater detail the piecewise autocorrelation determiner of Fig. 4: 
Figure 7 is a flowchart outlining the watermark retrieval process according to this invention: 
Figure 8 is a flowchart outlining in greater detail the global autocorrelation determination step of Fig. 7: 

20 Figure 9 is a flowchart outlining in greater detail the piecewise autocorrelation determination step of Fig. 7: 

Figure 10 shows retrieved digital watermarks using a first scanning resolution: 
Figure 11 shows retrieved digital watermarks using a coarse scanning resolution: and 
Figure 12 shows watermarks retrieved from the image shown on Fig. 1 using a constant shift. 

25 [0014] Fig. 1 shows an image 100 containing a hidden, or embedded, digital watermark. Fig. 2 shows a processed 
image 100', formed by processing the image 100 according to the method of this invention, to retrieve or extract the 
embedded watermark 110. The image 100 was selected because the image 100 reflects one of the most difficult 
situations for watermark retrieval. Specifically, the image 100 shown In Fig. 1 is an image of a chapel, created by a 
180x90 dpi stochastic halftone screen with an embedded invisible digital watermark, printed at 400 dpi on a black and 

30 white printer. The embedded digital watermark is an "X" logo. This particular stochastic screen is designed so that the 
left 90x90 pixel square is identical to the right 90x90 pixel square, except within the area specified by the shape of the 
X logo of the embedded watermark. Within the X logo, all corresponding pairs between the left and right squares are 
conjugates. 

[001 S] In the image 100' shown in Fig. 2, the presence and clarity of the embedded digital watermark 110 retrieved 
35 from the original watermark-encoded image 100 of Fig. 1 can be seen, presuming the image has remained in unaltered 
digital format since the watermark embedding process. Operating in the digital realm, if the watermark-embedded 
digital image 100 shown in Fig. 1 is copied, the original image and the copy can be digitally overlayed. The copy is 
then digitally shifted 90 pixels to the right relative to the original image. When the two images are then logically ANDed 
together, the embedded watermark 110 becomes clearly visible, as shown in the image 100' shown in Fig. 2. 
40 [0016] Thus, by using the method and system of this invention, retrieving watermarks from images that have not 
been outside the digital realm since the embedding process is straightforward and produces clear results. However 
retrieving the same watermarks after the image has been transferred to a printed copy is not trivial. 
[0017] However, it should be appreciated that this image is not limited in any way to a printed image. The only 
requirement for retrieving an embedded digital watermark is that the image on which the retrieval operation is performed 
-*5 was once encoded with an embedded digital watermark. The system and method of this invention works equally well 
on images that have been printed and subsequently scanned, that have been previously converted into a digital image, 
or that have been maintained in electronic form. 

[0018] Fig. 3 shows a watermark extraction device 200 for extracting embedded digital watermarks from an image 
according to this invention. As shown in Fig. 3, an image containing an embedded digital watermark is input from an 

so image input device 300 over a link 310 to the watermark extraction device 200. It should be appreciated that the image 
input device 300 can be any device that stores and/or generates an electronic version of the image. 
[0019] Thus, the image can be a printed hardcopy version of the image, and the image input device 300 can be a 
scanner that scans and outputs an electronic version of the image over the link 31 0 to the watermark extraction device. 
Furthermore, the scanner 300 and the watermark extraction device 200 can be elements integrated into a digital pho- 

55 tocopier. 

[0020] Similarly, the image input device 300 can be a server or other node on a local area network, a wide area 
network, an intranet, the Internet or any other distributed network. In this case, the image is already stored on the 
network in electronic form. Finally, the link 310 can be h wired or wireless link to the scanner or other image conversion 
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device or to the network that forms the image input device 300. Thus, the image input device 300 and the link 310 can 
be any known elements that are capable of supplying an electronic image to the watermark extractor device 200. 
[0021] As discussed above, the system and method of this invention works equally well on images that have not 
been transferred to hardcopy. In this case, the image is already in digital format and the image is ready for processing 

5 by the watermark extraction device 200. 

[0022] The watermark extraction device 200 includes an I/O interface 210, a controller 220. a memory 230 and an 
autocorrelator 240. An image is received from the image input device 300 via the link 310. The I/O interface 210 
forwards the input image data received from the image input device, under the direction of the controller 220 5 to the 
memory 230. The autocorrelator 240 processes the image based on determined global and piecewise autocorrelation 

io to retrieve the watermarks and form an image where the watermarks are visible from the input image. This resulting 
image is then output via the I/O interface 210 and a link 410 to an output device 400. It should be appreciated that the 
output device 400 can be any device that outputs or displays the resulting image data. 

[0023] As shown in greater detail in Fig. 4. the autocorrelator 240 functionally includes a global autocorrelation de- 
terminer 241, a. moving window selector 242, a piecewise autocorrelation determiner 243, a shifted image generator 

is 244. a watermark retriever 245 and an image normalizer 246. The global autocorrelator 241 inputs the input image 
from the memory 230 and outputs global correlation peaks to the moving window selector 242. The moving window 
selector 242 moves over the input image and outputs a plurality of portions of the input image, each portion corre- 
sponding to the size of the moving window and to a current position of the moving window. The piecewise autocorrelation 
determiner 243 inputs the plurality of portions and outputs a local autocorrelation for each portion. The shifted image 

20 generator 244 inputs the local autocorrelations and the plurality of portions and outputs a shifted image for each portion. 
The watermark retriever 245 inputs the plurality of portions and the corresponding shifted images and outputs a plurality 
of combined portions where the watermarks are visible. The image visualizer 246 combines the plurality of combined 
portions into an output image. 

[0024] With the image data residing in the memory 230, the global autocorrelation determiner 241 determines a 

25 global autocorrelation for the input image by searching for peaks in the autocorrelation of the input image, and deter- 
mines if the image is a halftone image. If the image is a halftone image, the global autocorrelation determiner 241 
estimates the size and orientation of the halftone screen that was used for generating the halftone image. Next, the 
moving window selector 242 selects and moves a moving window over the image to select portions of the image. The 
piecewise autocorrelation determiner 243 then determines, for each portion of the image selected by the moving window 

30 selector the localized autocorrelation of that selected portion of the input image. The shifted image generator 244 next 
generates, for each portion of the image selected by the moving window selector, a shifted image. The watermark 
retriever 245 then retrieves, for each portion of the image selected by the moving window selector, the embedded 
digital watermarks. The image visualizer 246 normalizes the resultant image for visualization. The resulting image is 
then stored in the memory 230. 

35 [0025] Fig. 5 shows the global autocorrelation determiner 241 of Fig. 4 in greater detail. The global autocorrelator 
241 includes a peak determiner 241 1 , a halftone determiner 241 2 and a halftone estimator 241 3. Global autocorrelation 
begins in the peak determiner 2411, which searches the image for correlation peaks. Upon detection of these peaks 
the halftone determiner 241 2 determines if the image is a halftone. If the halftone determiner 241 2 determines that the 
image is a halftone, the halftone estimator 241 3 estimates a size and orientation of the halftone. If the halftone deter- 

40 miner determines that the image is not a halftone, the halftone determiner 241 2 outputs a signal to the controller 220 
to halt processing of the input image. 

[0026] Fig. 6 shows the piecewise autocorrelation determiner 243 of Fig. 4 in greater detail. The piecewise autocor- 
relation determiner 243 includes a moving window positioner 2431. an image cropper 2432, a mean determiner 2433. 
a mean subtractor 2434, a local autocorrelation determiner 2435, a peak locator 2436, and a correlation determiner 

45 2437. Once the moving window has been selected by the moving window selector 242, the piecewise autocorrelator 
determiner 243 begins by moving the selected window across the input image. As the moving window positioner 2431 
moves the selected window across the image, for each new window position, the image cropper 2432 crops the image 
to the portion of the image within the moving window. Next, the mean determiner 2433 determines the mean of that 
portion of the image the image. Then, the mean subtractor 2434 subtracts the mean from that portion of the image. 

50 Next, the local autocorrelation determiner 2435 determines a local autocorrelation of that portion of the image. Then, 
the peak locator 2436 locates a local peak for that portion of the image near a point estimated by the global autocor- 
relation determiner 241 . Finally, the correlation determiner 2437 determines the local maximal correlation of that portion 
of the image. 

[0027] The resulting image can be a printed or copied version of the input image, and the output device 400 can be 
55 a printer. Similarly, the output device 400 can be a monitor which is capable of displaying an electronic version of the 
resulting image for viewing. Furthermore, the scanner 300. the watermark extraction device 200 and the output device 
400 can be elements integrated into a single device, such as a digital photocopier. 

[0028] Similarly, the output device 400 can be a server or other node on a local area network, a wide area network, 
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an intranet, the Internet or any other distributed network. In this case : the resulting image is transferred and stored on 
the network in electronic form. Finally, the link 410 can be a wired or wireless link to the output device 400 or any other 
image output or display device or to the network. Thus, the output device 400 and the link 410 can be any known 
elements that are capable of receiving and outputting or storing the resulting electronic image from the watermark 
5 extraction device 200. 

[0029] Fig. 7 outlines a method according to this invention for retrieving embedded watermarks from images that 
have been converted to a printed copy. Beginning in step Si 000, control continues to step S1100. where the printed 
copy image is scanned. It should be appreciated that : if the image is already in electronic format, control jumps directly 
to step S1200. 

w [0030] In step S1200. the image is analyzed to determine if the image is a halftone image and to estimate a global 
autocorrelation for the image. Next, in step S1300, a moving window is selected. The size of the moving window is 
based on the estimation of the global autocorrelation analysis. Then, the image is further analyzed by iteratively applying 
the moving window over the entire image. At the beginning of each iteration, in step S1400, a next moving-window- 
sized portion of the image is selected. Then in step S1500 : the piecewise localized autocorrelation for each selected, 

is moving window-sized portion of the image is determined. Control then continues to step S1600. 

[0031] In step S1600, based on the results of the localized autocorrelation determination, an esfimate of the local 
peak is determined for each selected portion of the image. A shified image wi/l be generated for each selected portion 
of the image based on the peak value of the localized autocorrelation determined for that selected portion. Next, in 
step S1700, the embedded watermark is retrieved. Then, in step S1800, the data containing the selected portion of 

20 the image with the retrieved watermarks is normalized and stored into the memory for later visualization. In step S1 900 
the control routine determines if the entire image has been selected and analyzed. If not, control jumps back to step 
S1400. Otherwise, if the entire image has been analyzed, control continues to step S2000. In step S2000. the control 
routine stops. 

[0032] The resolution of the input device does not need to match the resolution of the input printed copy image. 
25 Importantly, the resolution of the input device can be lower than the resolution of the printed copy image. As discussed 
below, the system and method of this invention are capable of successfully detecting watermarks from images that 
were printed at 400dpi and scanned at 300dpi. Furthermore, if the resolution of the input device used to scan the printed 
copy image increases, the signal-to-noise ratio increases and the contrast of the retrieval watermark in the resulting 
image is enhanced. 

30 [0033] Due to the periodicity of the halftoning process, the global autocorrelation of a halftone image determined in 
step S1200 presents peak values at certain positions. If the image has remained in unaltered digital format since the 
watermark embedding process, these autocorrelation peaks are located exactly as a two dimensional comb function. 
For example, the halftone image 100 shown in Fig. 1 was generated by a stochastic halftone screen with the periodicity 
of 90 pixels in both x and y direction. Therefore, the autocorrelation peaks of image 100 are shown as a two dimensional 

35 comb function with 90 pixel separation in both the x and y directions. To determine the existence of this comb function 
and its periodicity and orientation, autocorrelation peaks other than the one at (0, 0) position are searched for. For the 
example image shown in Fig. 1 , two autocorrelation peaks at (90, 0) and (-90, 0) are located on the horizontal axis 
and two peaks at (0, 90) and (0, -90) on the vertical axis. 

[0034] However, if the halftone image has been converted into hardcopy format, i.e., is moved out of the digital realm, 

-io the printing and/or copying process, as well as the scanning process to convert the halftone image back to the digital 
realm, may introduce unknown scaling, rotation, distortion and noise into the reformed digital image. For example, the 
halftone image 100 shown in Fig. 1 was printed by a black and white printer at 400 dpi and scanned by a scanner also 
at 400 dpi. Theoretically, four peaks of the autocorrelation function on the horizontal and the vertical axes should remain 
at (90. 0), (-90, 0). (0, 90) and (-90, 0) locations. When searching for the actual global autocorrelation, two correlation 

•*s peaks are located near the horizontal axis at (89, 1 ) and (-89. 1 ), and two peaks near the vertical axis at (-1 , 90) and 
(1. 90). Therefore, if the embedded watermarks are assumed to have been generated by a stochastic screen with a 
horizontal arrangement, as described above, searching for localized correlation peaks by the piecewise autocorrelation 
determiner 243 can be reduced to searching only around point (89, 1 ) of each 90x90 pixel portion of the image. 
[0035] It should be appreciated that this search process can be straightforwardly varied to encompass embedded 

50 watermarks that have a vertical orientation or even a plurality of orientations. Furthermore, the system and method of 
this invention encompasses retrieving digital watermarks from color images. By using the same halftone screen for 
color prints, i.e., for each color separation layer of a CMYK image, detecting the embedded watermarks in a color 
image is performed identically to the process outlined above. However if a different halftone screen is used for each 
color separation layer, the retrieval process must be performed independently on each color separation layer. 

55 [0036] The moving window used in step S1 300 should be large enough to cover an area containing a portion of two 
adjacent correlated halftone patterns. For example, for the image 100 shown in Fig. 1, the moving window could vary 
from as small as 100x20 pixels to as large as 256x256 pixels. Larger moving windows provide a higher signal-to-noise 
ratio and faster speed in the localized piecewise autocorrelation determination performed in step S1500. In contrast, 
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smaller moving windows provide better results when the input image suffers from severe distortion. However smaller 
windows slow the piecewise localized autocorrelation determination. In processing the example image 100 illustrated 
in Fig. 1. with the printer and scanner resolutions both at 400 dpi. an arbitrary moving window size of 100x50 was 
selected. However it should be appreciated that the "moving" window can be at least as large as the full image from 
5 which the watermarks are to be retrieved, with the aforementioned drawbacks. 

[0037] It should be appreciated that, in step S1600 : the generation of a shifted image can be accomplished using a 
convention method. However, in the preferred embodiment to generate the shifted image : for each separate window 
positioned at a horizontal position i and a vertical position j : the shifted image for that window is subtracted from the 
input image for that window to retrieve the watermarks. That is: 

w 



where: 



75 



G res (i : j) is the resulting image data for the location (i.j) in which the watermarks are visible: 
G shifl (i : j) is the shifted image data at original location (L j): and 
G (L j) is the original image data at original location (i : j). 

20 [0038] Fig. 8 outlines in greater detail one method for performing the global autocorrelation determination of step 
S1200 of Fig. 7 according to the invention. Control commences in step S1200, and continues to step S1210. In step 
S1210. the global peaks in the input image are searched for. 

[0039] Next, in step S1 220, a determination of whether the image is a halftone is made by searching for local relative 
peaks other than the DC term. If the image is a halftone, at least two peaks of its autocorrelation, symmetric about the 
25 origin (0. 0) of the halftone source, should stand fairly above the average value of their neighborhoods in relation to a 
measure of the absolute value. If the image is not a halftone, control jumps to step S1250, where control is returned 
to step S1900. If : in step Si 220, the image is determined to be a halftone image, control continues to step S1230. In 
step S1230. the size and orientation of the halftone image is estimated. Next, in step S1240, control is returned to step 
S1300. 

30 [0040] Fig. 9 outlines in greater detail one method for determining the piecewise autocorrelation for the image of 
step S1500 of Fig 4 according to this invention. The moving window was selected in step SI 300. The current moving- 
window-sized portion of the image was selected in step S1400. In particular, in step S1400, the selected window is 
positioned at an initial location (i. j) within the input image and the input image for the current position of the moving 
window is cropped to the size of the moving window. Thus, the piecewise autocorrelation commences in step S1500, 

35 and proceeds to step S1510. In step SI 510, the mean of the cropped image for the current position of the moving 
window is determined. Then, in step S1 520, the mean for the current position of the moving window is subtracted from 
the cropped input image. Next, in step S1530, the autocorrelation of the cropped and mean-subtracted image is de- 
termined. Control then continues to step S1540. 

[0041] In step S1 540, a local autocorrelation peak is searched for near the point estimated by the global autocorre- 
40 lation determination of step S1200. Then, in step S1550, a threshold for the noise level is compared to determine the 

position of the local relative peak. The threshold for the noise level is approximately 2.0o , where a is the root mean 

square of the autocorrelation calculated for the current window position excluding autocorrelation at (0 : 0) and its 

immediate neighbors. If the peak value is lower than the threshold, control continues to step S1560. Otherwise, if the 

peak is greater than the threshold, control jumps to step S1570. 
45 [0042] In step S1560, the global autocorrelation estimate from step SI 200 is used for the current window position 

for later processing in step S1600. In step S1570, parabolic interpolation is used to estimate the peak. Control then 

continues to step S1580. In step S1580, control returns to step S1600 of Fig. 7. 

[0043] In step S1 570, parabolic interpolation is used to estimate an accurate maximal autocorrelation position. Pref- 
erably, the parabolic interpolation is defined by: 



50 



55 



V 1 V " f (i p -1j p )]/[2-f(i p J p ) - Ki p +"M P )- f(i p -1-i p )L 
yacc = i P +0 -MUi p V^ 

where: 
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f(i. j) is the calculated autocorrelation function for the current portion (i 5 j) : 
(i n: j n ) is the peak position within the current portion, and 

(x acc . y acc ) is an estimation of an accurate maximal position within the current portion. However it should be ap- 
preciated that there are other methods of performing this interpolation. 

[0044] In step S1 600. with the estimated peak position by step S1 500. the shitted version of the scanned image can 
be generated by: 

G Shift(' i) = W 1 G <' + lnt < X acc) J + int <yacc)> + W 2 G (' + int < X acc) + 1 « 

j + int(y acc )) + w 3 .G(i + int(x acc ),j + int(y acc ) + 1) + 
w 4 G(i +int(x acc ) + 1 .j + int(y acc ) + 1 ), 

where: 

Wt = {1.0 - [x acc - int(x acc )]} {1+[y acc -int(y acc )]} : 

w 2 = ( x acc- |nt ( x acc)l I 1 " tyacc" jnt (yaccl} 
w 3 - I ' 0-[X acc - 'nt(x acc ))}.[y acc -int(y acc )] : 

w 4 = l x acc" 

int (x acc )] [y a cc-int(y acc )] and 
G(i ; j) is the input image. 

The shift values x acc and y acc are determined from the piecewise autocorrelation determination for the corresponding 
current portion, where both (i, j) and (i+x acc , j+y acc ) are covered. 

[0045] Figs. 10 and 11 show examples of embedded watermark retrieval as outlined above in Figs. 5-7. The above- 
outlined method was conducted on a halftone image, printed by a 400 dpi printer and scanned in both 300 and 400 
dpi modes. Fig. 1 0 illustrates the image 500 resulting from performing the method of this invention on a 400 dpi printed 
image that was scanned at 400 dpi. The recovered watermarks 510 are clearly visible. Fig. 11 illustrates the image 
600 resulting from performing the method of this invention on a 400 dpi printed image that was scanned at 300 dpi. 
The recovered watermarks 610 are also clearly visible. 

[0046] By comparison, as shown in Fig. 1 2. efforts to retrieve watermarks by applying a constant shift determined by 
the global autocorrelation to the entire image 100 resulted in an image 700, which was both printed and scanned at 
400 dpi. Note the X logo watermarks 710 are only clearly visible on the left hand portion of the image shown in Fig. 12. 
[0047] As shown in Fig. 3, the watermark extraction device 200 is preferably implemented on a programmed general 
purpose computer. However, the watermark extraction device 200 can also be implemented on a special purpose 
computer, a programmed microprocessor or microcontroller and peripheral integrated circuit elements, an ASIC or 
other integrated circuit, a digital signal processor, a hardwired electronic or logic circuit such as a discrete element 
circuit, a programmable logic device such as a PLD, PLA. FPGA or PAL, or the like. In general, any device, capable 
of implementing a finite state machine that is in turn capable of implementing the flowcharts shown in Figs. 5-7, can 
be used to implement the watermark extraction device 200. 

Claims 

45 

1. A method for recovering an embedded invisible digital watermark from an image, comprising: 

determining a global autocorrelation of the image: 
selecting a window having a determined size: 
50 selecting a current portion of the image corresponding to the size of the window: 

determining a piecewise localized autocorrelation of the image for the current portion: 
generating a shifted image for the current portion: 
retrieving the embedded watermarks for the current portion: 

repeating the portion selecting, determining, generating and retrieving steps for a new current portion until the 
55 entire image has been analyzed: and 

generating a new image containing the retrieved watermarks. 

2. The method of claim 1 , wherein determining the global autocorrelation comprises: 
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searching the image for autocorrelation peaks: 
determining if the image is a halftone: and 
estimating a size and orientation of the halftone. 

The method of claim 1 or claim 2. wherein selecting the window comprises determining the window size, the 
determined window size being large enough to determining the window size : the determined window size being 
large enough to ensure a portion of two adjacent correlated halftone patterns is contained within the window. 

The method of any of claims 1 to 3 : wherein selecting the portion of the image corresponding to the size of the 
window comprises cropping the image at the current corresponding to the size of the window comprises cropping 
the image at the current position to the size of the window. 

The method of any of claims 1 to 4, wherein determining the localized autocorrelation for the current position 
comprises: 

determining a mean of the current portion: 
subtracting the mean from the current portion: 
determining a local autocorrelation of the current portion: 

locating a local peak of the current portion near a point estimated by the global autocorrelation: and 
determining a maximal correlation between the image and the shifted image for the current portion. 

The method of any of claims 1 to 5 : wherein generating the shifted image comprises: 

G shift(U) = w r G(i + int(x acc )J + int(y acc )) + w 2 -G(i + int(x acc ) + 1, 
j + int(y acc )) + w 3 .G(i + int(x acc ) ; j 4- int(y acc ) +1) + 
w 4 -G(i + int(x acc ) + 1,j + int(y acc ) + 1).. 

where: 

W1 = {1.0 - [x acc - int(x acc )H(1 +[y aC c - «nt(y a cc)]} : 
w 2 = [x acc - int(x aoc )]-{[1 -[y acc - int(y acc )]}: 
w 3 = {1 .0 - (x acc - int(x acc fi *[y acc - int(y acc )]: 
w 4 = [x acc - «nt(x acc )] -[y acc - int(y acc )]: and 
G(i t j) is the input image: and 

shift values x acc and y acc are determined from the piecewise autocorrelation determination for the correspond- 
ing current portion, where both (i : j) and (i+x acc , j+y acc ) are covered. 

7. A recovery system that retrieves embedded invisible digital watermarks from an image comprising: 

input means for inputting the image: 

global autocorrelation means for determining a global autocorrelation for the image: 

piecewise autocorrelation means for determining a piecewise autocorrelation for each of a plurality of portions 
of the image: 

shifting means for generating a shifted version for each of the plurality of portions of the image based on the 
piecewise local autocorrelation: 

new image generation means for generating, based on the plurality of portions of the corresponding shifted 

versions, a new image containing retrieved watermarks: and 

output means for displaying the new image containing the retrieved watermark. 

8. The system of claim 7, wherein the global autocorrelation means further comprises: 

searching means for searching the image for peaks: 
determining means for determining if the image is a halftone: and 
estimating means for estimating a size and orientation of the halftone. 
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9. A system that extracts a digital watermark from an image based on global and piecewise autocorrelation compris- 
ing: 

a controller: 
a memory: and 
an autocorrelator. 

10. The system of claim 9. wherein the autocorrelator comprises: 

a global autocorrelator that determines a gtobal autocorrelation of the image: 
a moving window selector that selects a moving window; 

a piecewise autocorrelator that determines a piecewise local autocorrelation of the image for each of a plurality 
of portions of the image: 

a shifted image generator that generates a shifted version for each portion based on the corresponding piece- 
wise autocorrelation: 

a watermark retriever that retrieves a watermark for each portion based on that portion and the corresponding 
shifted image: and 

an image visualizer that generates, based on the retrieved watermarks : a new image containing the retrieved 
visible watermarks. 
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